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ABSTRACT 

Beginning in fall 1991, Napa Valley College (NVC) , in 
California, switched from essentially mandatory placement of incoming 
students to an advisory, self-selection system, where students 
receive course recommendations based on assessment test results. In 
order to evaluate the validity of NVC*s assessment procedures, a 
study was conducted of assessed students' self-placement and success 
rates for fall 1991 and spring 1992. Students assessed prior to these 
semesters were divided into two groups, "advised" students making 
course selections based on their assessment test scores, course 
recommendations, and personal factors (e.g., high school grades, work 
schedule, and motivation), and a "blind" group utilizing only 
personal factors. The study then t:xamined differences in the level of 
math and English courses selected by the two groups and subsequent 
success in two English and two math courses. Major findings included 
the following: (1) 511 of the 184 students who enrolled in English 
courses without knowing their assessment outcomes enrolled at a level 
that would have been recommended, while 6A7. of the 220 advised 
students did so; (2) 69*4 of the 228 students who enrolled in math 
courses without knowing their assessment outcomes enrolled at a 
recommended level, while 77% of the 285 advised students did so; (3) 
at most, 27/i of the unadvised students enrolled in higher level 
courses than would have been recommended; (4) for the two math 
courses, there was no difference in success between the two groups of 
students; and (5) while there was no difference in success rates for 
one of the English courses, the other shewed a success rate of 35% 
for unadvised students, compared to 58'/. of advised students. In 
general, the findings alleviated NVC staff concerns concerning the 
self-selection placement system. (BCY) 
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Course Enrollments and Subsequent Success 
After Being Advised of or 'Blind' to 
Assessment Test Scores and Course Recommendations 

Penny Y. Jue 
Napa Valley College 



In Fall 1991 and Spring 1992, data were collected on students who completed re<"\dir,g, writing, 
or math assessment tests, then immediately enrolled in English or math courses. Students were 
randomly assigned to two conditions-advised or 'blind'— depending on which orientation they 
attended. The advised students (N=220 for English; N=?85 for math) received their 
assessment test scores and course recommendations. The blind students (N=184 for English; 
N=228 for math) placed themselves without their test scores and recommendations. Both 
advised and blind students were asked to consider several non-test measures when selecting their 
courses. The main purpose of this design was to create greater variability among test scores in 
the classes, which would be necessary to validate the tests. In addition, we were able to 
compare the enrollments and subsequent success of students who were advised of or blind to 
their test scores and course recommendations. Beginning in Fall 1991, the college switched 
from what had been essentially mandatory placement to an advisory, self-selection system. 
Some staff were concerned that under advisory placement, large numbers of students would 
disregard the placement system by enrolling into higher level courses than recommended; these 
fears were unfounded. The results indicated that the majority of advised students followed their 
placement advice, and the majority of blind students enrolled into the courses to which they 
would have been recommended. However, a comparison of the two groups yielded a significant 
difference in their English course enrollments, with a larger percentage of blind students placing 
themselves into higher-level courses than recommended. The subsequent success/non-success 
of the groups was significantly different in one English course level but not another. There was 
no significant difference between the groups in their enrollments into recommended courses for 
math, nor was there any significant difference in the success of advised or blind students in two 
math courses. The results were instrumental in gaining greater instructor acceptance of the 
advisory, self-selection placement system and helping to alleviate concerns over students who 
choose to place themselves into higher level courses than recommended. 



Introduction 



The Napa Valley College Matriculation Steering Committee commissioned research studies on 
the assessment tests used for placement into reading, writing, and math courses. Representatives 
from the Counseling, Language Arts, and Science/Math Divisions worked with the Research 
Analyst to design the studies. The studies were conducted to comply with State Matriculation 
Regulations for research on the validity of assessment tests and placement systems used in 
community colleges. In addition, the data would provide college divisions with information to 
evaluate the effectiveness of their current tests and course recommendations. 

This particular report focuses on student placement into English 120 and English 100 courses 
using the College Board APS Reading test and a locally developed writing sample assessment. 
English 120 is the first semester college level reading and composition course. English 100 
immediately precedes English 120 and prepares students for college writing. We also focused 
placement into Math 90 (Elementary Algebra) and Math 94 (Intermediate Algebra) using the 
UC/CSU Mathematics Diagnostic Testing Project (MDTP) instruments. We did not focus on 
other English and Math courses because enrollments were too small to analyze. 

In a proper validation study, it is best to evaluate an assessment instrument or procedure not 
presently used for placement. However, the procedures under investigation were being used in 
this fashion. This, coupled with a past history of mandatory placement with a waiver system 
led to expectations that the variability of test scores would be limited, which would then limit 
validity. Therefore the assessment test research committee decided to suspend the reporting of 
assessment scores a.'?d course recommendations for a portion of students. Some students would 
be advised of their test scores and course recommendations, while others would place themselves 
in courses without this information. We could then compare course enrollments and ultimate 
success of the advised students compared to the students who were 'blind' to their test scores 
and course recommendations. 



Method 

We collected data in Fall 91 and Spring 92 on students who completed assessment tests, then 
immediately enrolled in English 120, English 100, Math 94 or Math 90 courses. Students 
included in this study were limited to those assessed within several weeks prior to the beginning 
of the semester. Since the purpose of this research is to evaluate the assessment test procedures, 
this study focuses on students placed using those assessments, not on those ^ 'ho completed pre- 
requisite courses. 

Students were randomly assigned to the two conditions-advised or 'blind'-depending on which 
orientation they attended. The advised students received their assessment test score and course 
recommendations. The blind students placed themselves without their test scores and 
recommendation but could receive their test scores if they felt they needed them. (Few blind 
students chose to join the advised group.) Students, counselors and instructors were not aware 
of students' test scores. Both advised and blind students were asked to consider several non-test, 
self-reported measures when selecting their courses. 



Factors Students Were to Consider When Selecting Their Courses 



Advised Students 

• Assessment test score and course 
recommendations 

• Grades received in last Math and 
English courses 

• How recently they had taken other 
courses in the area 

• High school grades and how long 
since they attended high school 

• Work schedule 

• Family responsibilities 

• Study skills 

• Their own motivation to succeed in 
courses 



Blind Students 



• Grades received in last Math and 
English courses 

• How recently diey had taken other 
courses in the area 

• High school grades and how long 
since they attended high school 

• Work schedule 

• Family responsibilities 

• Study skills 

• Their own motivation to succeed in 
courses 



The designation of a student as 'blind' or 'advised' had to be manually-entered onto the 
computer. To minimize labor, we entered these designations only for students who 
completed a writing sample in addition to the reading test. Students were recommended to 
English courses based on the higher of their scores on the reading or writing test. 



Did Advised Students Tend to Follow English Course Recommendations? 
Where Did Blind Students Place Themselves? 



English Course Enrollments 





Below 
Recomm. • 


At 
Recomm. 


Above 
Recomm. 


Total* 


Advised 


26% 


64% 


11% 


220 


Blind 


23% 


51% 


26% 


184 



• Close to two-thirds of the advised students' enrollments were in courses recommended. 

• Over half of the blind student enrollments were in courses to which they would have 
been recommended. 

• There was a statistically significant difference between the advised and blind students in 
their course placements [X^(2) = 17.2, p < ,001]. A larger percentage of Blind student 
enrollments were in courses above those to which they would have been recommended. 



*The actual number of enrollments is higher. These totals are limited to students for whom 
we had complete records on reading scores, writing scores, and advised/blind status. 



Did the Groups Differ in Their Success or Non-Success? 



They did in one English course level but not another. 

English 120 
Freshman Composition 

Not 





Successful 
(A, B, C, CR) 


Successful 
(D, F, NC) 


Withdrew 


Total 


Advised 


66% 


9% 


26% 


160 


Blind 


58% 


18% 


24% 


98 



• There was no significant difference between the success or non-success of the groups. 



English 100 
Preparation for Freshman Composition 



Not 





Successful 
(A. B, C, CR) 


Successful 
(D, F, NC) 


Withdrew 


Total 


Advised 


58% 


19% 


24% 


139 


Blind 


35% 


25% 


40% 


68 



• There was a statistically significant difference between the success of the groups 

[X^(2)=9.4, p< .01]. Advised students were more successful, while a larger percentage 
of Blind students withdrew from the course. 



Did Advised Students Tend to Follow Math Course Recommendations? 
Where Did Blind Students Place Themselves? 



Math Course Enrollments 





Below 
Recomm. 


At 
Recomm. 


Above 
Recomm, 


Total 


Advised 


8% 


77% 


15% 


285 


Blind 


14% 


69% 


17% 


228 



• Over three-fourths of the advised students' enrollments were consistent with their 
placement advice. 

• Two-thirds of the blind students' enrollments were in the course(s) to which they would 
have been recommended. 

• There was no significant difference between the blind and advised groups in their course 
placements. 



s 



Did the Groups Differ in Their Success or Non-Success? 

In the two math courses, the Advised and Blind groups did not differ significantly in their 
success. 



Math 90 
Elementary Algebra 





Successful 
(A, B, C, CR) 


Successful 
(D, F, NC) 


Withdrew 


Total 


Advised 


53% 


21% 


26% 


145 


Blind 


52% 


23% 


15% 





Math 94 
Intermediate Algebra 

Not 





Successful 
(A, B, C, CR) 


Successful 
(D, F, NC) 


Withdrew 


Total 


Advised 


57% 


16% 


27% 


99 


Blind 


59% 


20% 


22% 


41 i 
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We can compare some of the assumptions staff had made about what would happen to actual 
research findings. 



What They Said 
Would Happen 



• Over 90% of our students follow 
placement advice. 



• Students won't want to place 
themselves without testing infor- 
mation. They need those scores. 



• Blind or advisory placement would 
lead large numbers of students to 

misplace" themselves. 

• Students overestimate their abilities in 
English and underestimate their 
abilities in math. 



What Really 
Happened 

• Anywhere from half to three-fourths 
of students followed placement advice, 
depending on the course subject. Up 
to one-fourth of students placed 
themselves in courses below those 
recommended. 

• Most of the 'blind' students- who 
placed themselves without test results 
or course recommendations- 
voluntarily chose to remain in the 
blind group. 

• Even among the 'blind' students, at 
most 27 % enrolled in higher level 
courses than they would have been 
recommended, 

• Greater percentages of students (both 
advised and blind) followed course 
recommendations for math than they 
did for English, and greater 
percentages placed themselves in 
lower level English courses than 
recommended. 



^0 



Conclusions 



• The majority of students enrolled in courses recommended or, in the case of blind students, courses 
to which they would have been recommended. We did not find large percentages of students placing 
themselves in higher level courses; in fact, some placed themselves in lower level courses. 

• Students are able to select their courses without test scores or recommendations. However, their 
success rates differed depending on the course; advised student showed significantly greater success 
in one English course but not another. There were no significant differences in student success in 
the two math courses. In any event, we did not find blind students systematically performing more 
poorly than those advised. 

These findings had impact on staff attitudes toward assessment. 
Impact of Findings 

• Instructors are more accepting of an advisory self-selection placement system. 

• We were able to alleviate concerns over students who chose to place themselves into highe/ level 
courses than recommended. 

• Instructors are much more open to evaluating assessment procedures and basing decisions on actual 
findings. 



